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LISTING OF CLAIMS: 

Claim 1 (Currently Amended): An information input processing computer system for mapping 
gestures to keys of a virtual invisible keyboard, the system comprising one or several cameras, 
one or more memories with CPU connected to the cameras, and processes running in the CPU 
that associate gesture movements ™«Hr in a touch-sensor free environment with typing and 
produce gesture associated textual output, wherein said processes capture gesture images, 
classify each types type of gesture image into a respective one of a plurality of classes depending 
on the type of gesture, and associate each of the classes with one or more possible keys pfthe 
invisible keyboard, assign a probability to each of said possible keys, and integrate the 
probabilities assigned to the possible keys to identify a word for a sequence of gestures. 

Claim 2 (Original): The gesture-key mapping system as in claim 1, where a feedback is 
provided to the user on what kind of keys are associated with the user's gestures. 

Claim 3 (Original): The gesture-key mapping system as in claim 2, where the feedback is 
provided using one or more of the following: displaying keys on a display, playing sounds labels 
for keys, displaying image indicators on a display, playing special sound indicators, projecting 
the keyboard to any surface, and displaying picture of the keyboard with user's hands. 
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Claim 4 (Currently Amended): An information input processing, gesture-key mapping computer 
8y8te m «~ text fro m h„nd gestu res of a u ^ r* fo« to an invisible keyboard, the 

system comprising one or several cameras, one or more memories with CPU connected to the 
cameras, and processes running in the CPU that associate gesture movements with typing and 
produce gesture associated textual output, where the gesture-key processing is provided using the 
following modules: 

a) a gesture capturing module that captures gestures relative, to an invisible 

and mH- ™ « tnuch-sensor free environment, through camera 

sensors; 

b) b) a gesture classificator module that classifies each type of gesture into a 
respective one of a plurality of classes depending on the type of gesture 
movements; 

c) an associate: module for associating gesture classes or sequence of gesture 
classes with one or more possible keys of the invisible keyboard and assigns a 
probability to each of said possible keys; and 

d) an integrator module that integrates the probabilities assigned to the possible 
keys to identify a word for a sequence of gestures. 

Claim 5 (Original): The system as m claim 4, where the integrator module includes one or more 

of the following: 

a) language module component that estimate probabilities of word strings 

corresponding to key candidate sequences; 
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b) character frequency module that estimate probabilities of character strings 
corresponding key candidate sequences; 

c) confusable matrix that estimate how often correct gesture classes are 
confusable with another gesture classes; 

d) gesture classes probability model that estimate probability of observing a string 
of gesture classes given a sequence of gesture frames; . 

e) computation of a probability of production a sequence of keys given a string of 
gesture frames; 

f) generation of a lattice of sequences of keys given sequence of gesture frames; 

g) finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 6 (Original): A system according to Claim 5, wherein each sequence of keys receives a 
probability score, and the sequences of keys that receive low scores are removed and are not 
continuing when new candidates for keys arrive. 

Claim 7 (Currently Amended): A system according to Claim 4 4, further comprising a gesture 
correlator module to allow to adjust automatically as the invisible keyboard to changes in the 
hand positions of the user, and wherein the gesture correlator module ac ts between the gesture 
classificator module and the associator module to maintain a consistent mapp ing of gesture 
classes to keys of the invisible keyboard despite said ch anges in t he hand posi tions of the user. 
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Claim 8 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gesture relative to a virtual, invis i ble keyboard made in a touch-sensor tree 
environment and without the presence of a real keyboard and the gestures are associated with the 
most probable keys that would be typed if a keyboard were presented, said method including the 
steps of using a computer system to map gestures made in a touch-sensor free environment to 
keys of a toe virtual keyboard, including the steps of running processes on the computer to 
capture gesture images, to classify each type of gesture image into a respective one of a plurality 
of classes depending on the type of gesture, and to associate each of the classes with one or more 
possible keys th ? invisible keyboard, assign a probability to each of said possible keys, and 
integrate the probabilities assigned to the possible keys to identify a word for a sequence of 
gestures. 

Claim 9 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gestures relative to a virtual, invisible keyboard and without the presence of a real 
keyboard and the gestures are associated with the most probable keys that would be typed if a 
keyboard were presented, said method including the step of using a computer system to map 
gestures made in a touches™- free environment to keys of a the virtual keyboard, including the 
step of running processes on the computer to capture gesture images, to classify each type of 
gesture image into a respective one of a plurality of classes depending on the type of gesture, and 
to associate each of the classes with one or more possible keys of the invisible Keyboard, assign a 
probability to each of said possible keys, and integrate the probabilities assigned to the possible 
keys to identify a word for a sequence of gestures, and wherein the probability is computed using 
HMM. 

j G:Vnwn\10S\13317\amaidM3317.am5.doc 

~ „,„ht Ttm*l - sVR-UBPTO-EFXRF-1i1 • DN1B:8729308 • CSID:3167424386 • DURATION (mn>«):07.02 
PACE 10/21 ■ RCVD AT 8/1 a/2004 4:10:59 PM pastern Dayligm Tlmel svr.UBPTo-efxkp "™ 



8-18-04; 3:06PM;SSMP FAX 



; 51 ©742*366 



# 11/ 2 1 



Claim 10 (Currently Amended): A method of typing using a virtual keyboard having a multitude 
of virtual keys, comprising the steps: 

pairing typing gestures relative to a virtual, invisible keyb oard made in a touch- 
sftnsnr free environment and without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output 
including the step of running processes on a computer to capture gesture images, classify each 
type of gesture image into a respective one of a plurality of classes depending on the type of 
gesture, and to associate each of the classes with one or more possible keys of the invisible 
keyboard, assign a probability to each of said possible keys, and integrate the probabilities 
assigned to the possible keys to identify a word for a sequence of gestures. 

Claim 1 1 (Original): A method according to Claim 10, wherein the typing gestures are made by 
a person, and further comprising the steps of providing feedback to the person on texture output 
associated with the gestures. 

Claim 12 (Original): A method according to Claim 11, wherein the step of providing feedback 
includes the step of displaying an image of typing keys associated with the gestures. 

Claim 13 (Cancelled). 
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Claim 14 (Currently Amended): A method of typing using a virtual feasible keyboard, 

comprising the steps: 

making typing gestures t?1 °*™ tn a virtual invisible keyboard made in a touch, 

sensor free environment and without any real keyboard; 
sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output: and 
wherein the producing steps Men includes the etep Mens of classifying each type of gesture into a 
respective one of aplurality of classes depending on the type of gesture, associating each of said 
classes with one or more possible keys of the invisible keyboard, assigning a probability to each 
of said possible keys, and integrating the probabilities assigned to the possible keys to 
identifying a word for a response sequence of gestures. 

Claim 15 (Original): A method according to Claim 14, wherein the producing step further 
includes the step of associating gesture classes with individual typing keys. 

Claim 16 (Previously Presented): A method according to Claim 14, farther comprising 
providing training data in words or sentences with certain timing data. 

Claim 17 (Currently Amended): A typing system using a viitoaljnvkibis keyboard, comprising 
means for sensing typing gestures made relative to a virtual, invit e ke yboard made in a touch- 
sensor free environment and without any real keyboard; and 
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means for producing, from the sensed typing gestures, gesture associated textual 
output said producing means including a computer and processes running on the computer to 
capture gesture images, to classify each type of gesture image into a respective one of a plurality 
of classes depending on the type of gesture, and associate each of the classes with one or more 
possible keys nf th e invisible keyboard, assign a probability to each of said possible keys, and 
integrate the probabilities assigned to the possible keys to identify a word for a sequence of 
gestures. 

Claim 18 (Original): A system according to Claim 17, wherein the typing gestures are made by a 
person, and farther comprising means for providing feedback to the person on texture output 
associated with the gestures. 

Claim 19 (Original): A system according to Claim 18, wherein the means for providing 
feedback includes means for displaying an image of typing keys associated with the gestures. 

Claim 20 (Cancelled). 

Claim 21 (Currently Amended): A typing system using a virtual invisibly keyboard, comprising 
means for sensing typing gestures made relative to a virtual, invisible keyboard 

made in a touch-sensor free environment and without any real keyboard; and 

means for producing, from the sensed typing gestures, gesture associated textual 

output; and 
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wherein the producing means includes means for classifying each type of gesture 
into a respective one of a plurality of classes depending on the type of gesture, associating each 
of said classes with one or more possible keys of the invisible keyboard, assigning a probability 
to each of said possible keys, and integrating the probabilities assigned to the possible keys to 
identifying a word for a response of gestures. 

Claim 22 (Original): A system according to Claim 21, wherein the producing means further 
includes means for associating gesture classes with individual typing keys.' 

Claim 23 (New): A method according to Claim 10, wherein the step of running processes on the 
computer includes the step of mapping the sensed typing gestures to keys of the keyboard based 
on a statistical machine that interprets sequences of typing like gesture classes as intended words 
based on user typing gesture models. 

Claim 24 (New): A method according to Claim 10, wherein gesture input is represented as wave 
forms that are digitized and clustered in gesture frames, and then processed by HMM machines 
that represent difference models for different typing patterns: and the step of ^imninfr processes 
includes the step of using two statistical components, a first statistical compo nent to map 
gestures to keys, and a second, language model component to map keys to letters and words . 
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